texlive_latest: init at 2023.20230606 #236382

apfelkuchen6 · 2023-06-07T01:36:45Z

Description of changes

This is an alternative approach to #222142 suggested by @xworld21 in #222142 (comment): This adds texlive-2023 in addition to the old version instead of replacing it.

I also noticed the following:

the procedure described in UPGRADING.md is no longer applicable since texlive.scheme-full.pkgs is empty since texlive.combine: move dependencies to attribute tlDeps, resolve them … #218331. As this will be resolved/changed by texlive: generate "fixed-hashes.nix" from Nixpkgs attribute #223104 anyway, I haven't bothered with updating the instructions yet.
dvisvgm's pdf conversion feature is incompatible with ghostscript >= 10. We have a test for that, but the test isn't good enough: It just successfully creates an empty svg file. The dvisvgm version shipped with texlive-2023 also has a mupdf-backend as a workaround. I'm not sure what to do about the old version.

Things done

drupol

Just added a few comments while passing by :)

pkgs/test/texlive/default.nix

pkgs/tools/typesetting/tex/luametatex/default.nix

pkgs/tools/typesetting/tex/texlive/bin.nix

pkgs/test/texlive/default.nix

xworld21 · 2023-06-11T19:34:29Z

pkgs/tools/typesetting/tex/texlive/bin.nix

How hard would it be to have bin.nix as before, and an extra bin-latest.nix with overrides only? Which I believe would be the more nixpkgs-y way of providing two versions.

Alternatively, bin-latest.nix could be a straight copy of bin.nix, which would become the new bin.nix once it becomes frozen.

Exactly as before is rather inconvenient as the sources would have to be overridden for every derivation. Just passing in the source as an argument and doing the patches and version-specific hacks via overrides shouldn't be too hard at this time as 2023 mostly just adds hacks over 2022. This might become more inconvenient if parts of hacks have to be removed.

I don't really like either approach. Using overrides makes upgrading less convenient and having a complete copy of the file feels really wasteful. I think just keeping bin.nix mostly general and put version-specific hacks/patches somewhere else for both versions is better as the upgrade remains just moving a file.

xworld21 · 2023-06-11T19:36:53Z

pkgs/tools/typesetting/tex/texlive/combine.nix

@@ -317,23 +317,29 @@ in (buildEnv {
  ''
    rm "$out"/bin/*-sys
    wrapBin
-  '' +
+  ''
+  + bin.cleanBrokenLinks +


(Incidental: cleanBrokenLinks will disappear with the bin container patch. I believe it is already a no-op since we started using texlinks.)

xworld21 · 2023-06-11T19:37:43Z

pkgs/tools/typesetting/tex/texlive/combine.nix

      substitute "$TEXMFDIST"/scripts/context/lua/mtxrun.lua mtxrun.lua \
        --replace 'cache_uuid=osuuid()' 'cache_uuid="e2402e51-133d-4c73-a278-006ea4ed734f"' \
        --replace 'uuid=osuuid(),' 'uuid="242be807-d17e-4792-8e39-aa93326fc871",'
      FORCE_SOURCE_DATE=1 TZ= faketime -f '@1980-01-01 00:00:00 x0.001' luatex --luaonly mtxrun.lua --generate
+
+      if [[ -e "$out/bin/luametatex" ]]; then
+        # we intentionally don't use the scripts.lst-mechanism here since we don't want to strip the suffix or wrap the scripts here.


(This should also become unnecessary after the bin container patch.)

pkgs/tools/typesetting/tex/texlive/default.nix

xworld21 · 2023-06-11T19:40:55Z

pkgs/top-level/all-packages.nix

@@ -5003,7 +5005,8 @@ with pkgs;
  texFunctions = callPackage ../tools/typesetting/tex/nix pkgs;

  # TeX Live; see https://nixos.org/nixpkgs/manual/#sec-language-texlive
-  texlive = recurseIntoAttrs (callPackage ../tools/typesetting/tex/texlive { });
+  texlive = recurseIntoAttrs (callPackage ../tools/typesetting/tex/texlive/frozen.nix { });


Picking up on the above, I believe tradition is to use default.nix for the stable version (thus default.nix -> make-tex-live.nix and frozen.nix -> default.nix).

There already are a lot of files here and I feel like this makes it even less obvious where what is as this breaks up the symmetry between the two versions and replaces default.nix with a mostly empty file just containing version information. How do you feel about the following layout:

├── bin.nix ├── combine.nix ├── generate-fixed-hashes.nix ├── latest │ ├── default.nix │ ├── fixed-hashes.nix │ ├── tlpdb.nix ├── make-texlive.nix ├── make-texlive-packages.nix ├── stable │ ├── default.nix │ ├── fixed-hashes.nix │ ├── tlpdb.nix ├── tl2nix.sed └── UPGRADING.md

Version specific stuff is put into subdirectories whereas make-texlive.nix becomes a generic function that builds the texlive(_latest) scope. The code that builds the texlive packages (currently tl in default.nix) is extracted into a separate file make-texlive-packages.

The entry points called in all-packages.nix are moved into the subdirectories and call make-texlive.nix, with the version-specific stuff, possibly applying overrides.

@apfelkuchen6 now that the binary containers are in, what do you think of refactoring the packages in texlivePackages? Presumably with tlpdb.nix and fixed-hashes.nix in a separate texlive-packages/ folder, while texlive keeps the combine machinery in place. The Nix expressions to build the packages should probably stay under texlive, e.g. texlive.buildPackage. Priority should be to make the entire thing easy to override, something which is currently very hard.

You have a few pieces here already – shall we start from those? Say with a new 0-rebuild PR?

xworld21 · 2023-06-11T19:42:16Z

pkgs/tools/typesetting/tex/texlive/latest.nix

+callPackage ./default.nix {
+  inherit version tlpdb tlpdbxz fixedHashes urlPrefixes src;
+  texlive = texlive_latest;
+}


E.g. could the 2023-specific overrides happen here? Likewise 2022-only patches could be specified in frozen.nix.

pkgs/tools/typesetting/tex/texlive/latest.nix

pkgs/tools/typesetting/tex/texlive/frozen.nix

xworld21

That's very good. I thought it would be a lot harder.

I like that frozen.nix and latest.nix more or less contain the essential information about each specific version. However, the logic is still duplicated around – default.nix has special code for 2022 and 2023; the urls can be computed once in default.nix, like before; and similar (see comments).

I suggest we push further, and have a make-tex-live.nix that takes as input: the version info, the tlpdb.xz hash (just the hash, not the url!), tlpdb.nix, the binary sources, and fixed hashes. Then frozen.nix calls that function and plugs in whatever overrides are required (e.g. version-specific patches).

This prepares the use of callPackage (to avoid repeating the parameter names all over the place) and makes static analysis possible

The assertion that all TeX Live packages have a fixed hash is time consuming and should only be checked when running tests.

this is purely a whitespace change that was separated from the previous commit to make the diff easier to read

…level

This generic interface allows the addition of texlive-2023 without repeating too much code or case distinctions TODO: rename default.nix -> generic.nix, stable.nix -> default.nix This will be done later as this makes rebasing a lot harder

github-actions bot added the 6.topic: TeX Issues regarding texlive and TeX in general label Jun 7, 2023

ofborg bot added the 8.has: package (new) This PR adds a new package label Jun 7, 2023

ofborg bot requested a review from veprbl June 7, 2023 03:09

ofborg bot added 10.rebuild-darwin: 101-500 10.rebuild-linux: 101-500 labels Jun 7, 2023

veprbl requested a review from xworld21 June 7, 2023 03:40

drupol reviewed Jun 7, 2023

View reviewed changes

apfelkuchen6 force-pushed the nixpkgs-multi-version branch from 32fcec8 to 4f07992 Compare June 7, 2023 14:13

xworld21 reviewed Jun 11, 2023

View reviewed changes

pkgs/test/texlive/default.nix Outdated Show resolved Hide resolved

xworld21 reviewed Jun 11, 2023

View reviewed changes

pkgs/tools/typesetting/tex/texlive/default.nix Outdated Show resolved Hide resolved

xworld21 reviewed Jun 11, 2023

View reviewed changes

pkgs/tools/typesetting/tex/texlive/latest.nix Outdated Show resolved Hide resolved

xworld21 reviewed Jun 11, 2023

View reviewed changes

pkgs/tools/typesetting/tex/texlive/frozen.nix Outdated Show resolved Hide resolved

xworld21 suggested changes Jun 11, 2023

View reviewed changes

apfelkuchen6 force-pushed the nixpkgs-multi-version branch from 4f07992 to f52763b Compare June 13, 2023 20:40

apfelkuchen6 marked this pull request as draft June 19, 2023 22:41

apfelkuchen6 and others added 10 commits August 21, 2023 22:41

texlive: move combinePkgs from default.nix to combine.nix

b9d147b

texlive.combine: explicitly list params

f6fb151

This prepares the use of callPackage (to avoid repeating the parameter names all over the place) and makes static analysis possible

texlive: factor out tlpdb overrides

e2e1127

tests.texlive.fixedHashes: init

60b9879

The assertion that all TeX Live packages have a fixed hash is time consuming and should only be checked when running tests.

texlive: make scope

dbbf86e

texlive: indent body of attrset

15a2fb4

this is purely a whitespace change that was separated from the previous commit to make the diff easier to read

treewide: access texlive packages via texlive.pkgs instead of the top…

10811c1

…level

texlive: extract generic interface

711e366

This generic interface allows the addition of texlive-2023 without repeating too much code or case distinctions TODO: rename default.nix -> generic.nix, stable.nix -> default.nix This will be done later as this makes rebasing a lot harder

texlive: fix cross

8aaefd9

luametatex: init at 2.10.08

11f4310

apfelkuchen6 added 4 commits August 21, 2023 22:41

texlive: extract patches specific to texlive-2022 into stable.nix

5506618

texlive: adjust generate-fixed-hashes for use with texlive_latest

afdc12a

texlive.combine: prepare for texlive-2023

6d7b6f0

texlive_latest: init at 2023.20230807

ea56a00

apfelkuchen6 force-pushed the nixpkgs-multi-version branch from 70053c0 to ea56a00 Compare August 21, 2023 20:54

apfelkuchen6 mentioned this pull request Aug 21, 2023

texlive: transform into overridable scope and extract generic interface #250626

Draft

12 tasks

chaoflow mentioned this pull request Sep 20, 2023

texlive: 2022 -> 2023 #222142

Closed

12 tasks

wegank added 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 2.status: merge conflict This PR has merge conflicts with the target branch labels Mar 19, 2024

stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 20, 2024

wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

texlive_latest: init at 2023.20230606 #236382

texlive_latest: init at 2023.20230606 #236382

apfelkuchen6 commented Jun 7, 2023

drupol left a comment

xworld21 Jun 11, 2023

apfelkuchen6 Jun 11, 2023 •

edited

Loading

xworld21 Jun 11, 2023

xworld21 Jun 11, 2023

xworld21 Jun 11, 2023

apfelkuchen6 Jun 11, 2023 •

edited

Loading

xworld21 Aug 5, 2023

xworld21 Jun 11, 2023

xworld21 left a comment

texlive_latest: init at 2023.20230606 #236382

Are you sure you want to change the base?

texlive_latest: init at 2023.20230606 #236382

Conversation

apfelkuchen6 commented Jun 7, 2023

Description of changes

Things done

drupol left a comment

Choose a reason for hiding this comment

xworld21 Jun 11, 2023

Choose a reason for hiding this comment

apfelkuchen6 Jun 11, 2023 • edited Loading

Choose a reason for hiding this comment

xworld21 Jun 11, 2023

Choose a reason for hiding this comment

xworld21 Jun 11, 2023

Choose a reason for hiding this comment

xworld21 Jun 11, 2023

Choose a reason for hiding this comment

apfelkuchen6 Jun 11, 2023 • edited Loading

Choose a reason for hiding this comment

xworld21 Aug 5, 2023

Choose a reason for hiding this comment

xworld21 Jun 11, 2023

Choose a reason for hiding this comment

xworld21 left a comment

Choose a reason for hiding this comment

apfelkuchen6 Jun 11, 2023 •

edited

Loading

apfelkuchen6 Jun 11, 2023 •

edited

Loading